System and method for synthetic aperture radar target recognition using multi-layer, recurrent spiking neuromorphic networks

ABSTRACT

A system configured to identify a target in a synthetic aperture radar signal includes: a feature extractor configured to extract a plurality of features from the synthetic aperture radar signal; an input spiking neural network configured to encode the features as a first plurality of spiking signals; a multi-layer recurrent neural network configured to compute a second plurality of spiking signals based on the first plurality of spiking signals; a readout neural layer configured to compute a signal identifier based on the second plurality of spiking signals; and an output configured to output the signal identifier, the signal identifier identifying the target.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit of U.S. Provisional PatentApplication No. 62/617,035, filed in the United States Patent andTrademark Office on Jan. 12, 2018, the entire disclosure of which isincorporated by reference herein. This application is also related toU.S. patent application Ser. No. 15/784,841, filed in the United StatesPatent and Trademark Office on Oct. 16, 2017, the entire disclosure ofwhich is incorporated by reference herein.

BACKGROUND

Synthetic aperture radar (SAR) is widely used for target imaging andtarget recognition utilizing radio frequency (RF) electromagnetic waves.In SAR systems, electromagnetic waves are emitted toward a target andthe reflected waves are collected by a radar antenna. Because SAR datais able to provide reflected RF signals from a target at a highresolution, the two-dimensional (2D) and/or three-dimensional (3D) shapeof the target can be computed from the raw SAR data.

Generally, existing techniques use the raw SAR data called phase historydata to form target images, such as bitmaps, and the target images areused for target recognition, such as by displaying the target images ona screen for viewing by a human operator. Forming target images from rawSAR data is a computationally intensive computing process, whichrequires significant computing power, which may make real-time targetrecognition difficult in many applications, especially where there areconstraints on the size, weight, and power (SWAP) of the hardware.

A spiking neuromorphic network is inspired by the human brain in that itprocess signals in the spiking domain, where all of the signals arerepresented by spiking sequences. On average, spiking neuromorphicnetworks consume very little energy because energy is only consumed whenthere is a spike and because, most of the time, there is no spike.

SUMMARY

Aspects of embodiments of the present invention relate to a system andmethod for recognizing and classifying targets in raw SAR data utilizingrecurrent, multi-layer spiking neuromorphic networks.

According to one embodiment of the present invention, a method foridentifying a target in a synthetic aperture radar signal includes:extracting, by a feature extractor, a plurality of features from thesynthetic aperture radar signal; encoding, by an input spiking neuralnetwork, the features as a first plurality of spiking signals; supplyingthe spiking signals to a multi-layer recurrent neural network to computea second plurality of spiking signals; computing, by a readout neurallayer, a signal identifier based on the second plurality of spikingsignals; and outputting the signal identifier from the readout neurallayer, the signal identifier identifying the target.

The plurality of features may include an amplitude of the syntheticaperture radar signal.

The plurality of features may include a phase of the synthetic apertureradar signal.

The readout neural layer may include a linear classifier.

The input spiking neural network, the multi-layer recurrent neuralnetwork, and the readout neural layer may be implemented by aneuromorphic chip.

The method may further include computing average spiking rates from thesecond plurality of spiking signals, wherein the signal identifier maybe computed based on the average spiking rates.

The multi-layer recurrent neural network may include: a first excitatoryneuron layer configured to receive the first plurality of spikingsignals from the input spiking neural network; a first inhibitory neuronlayer; a second excitatory neuron layer; and a second inhibitory neuronlayer connected to the second excitatory neuron layer, the firstexcitatory neuron layer being configured to supply spiking signals to:the first excitatory neuron layer; the first inhibitory neuron layer;and the second excitatory neuron layer, the first inhibitory neuronlayer being configured to supply spiking signals to the first excitatoryneuron layer, the second excitatory neuron layer being configured tosupply spiking signals to: the second excitatory neuron layer; thesecond inhibitory neuron layer; and the readout neural layer, and thesecond inhibitory neuron layer being configured to supply spikingsignals to the second excitatory neuron layer.

The input spiking neural network may include a plurality of inputneurons arranged in a grid, wherein the first excitatory neuron layermay include a plurality of first excitatory neurons arranged in a grid,the plurality of first excitatory neurons including a plurality ofcritical neurons uniformly distributed in the grid, and wherein theplurality of input neurons may be connected to the critical neurons tomaintain spatial relationships between the input neurons incorresponding ones of the critical neurons.

The first excitatory neuron layer may include a plurality of excitatoryneurons arranged in a grid, and a neuron of the first excitatory neuronlayer may be configured to supply spiking signals to neurons in a localneighborhood of the grid around the neuron.

The first excitatory neuron layer may include a plurality of firstexcitatory neurons arranged in a grid, the second excitatory neuronlayer may include a plurality of second excitatory neurons arranged in agrid, and the plurality of first excitatory neurons may be connected tothe second excitatory neurons to maintain spatial relationships betweenthe first excitatory neurons in corresponding ones of the secondexcitatory neurons.

According to one embodiment of the present invention, a systemconfigured to identify a target in a synthetic aperture radar signalincludes: a feature extractor configured to extract a plurality offeatures from the synthetic aperture radar signal; an input spikingneural network configured to encode the features as a first plurality ofspiking signals; a multi-layer recurrent neural network configured tocompute a second plurality of spiking signals based on the firstplurality of spiking signals; a readout neural layer configured tocompute a signal identifier based on the second plurality of spikingsignals; and an output configured to output the signal identifier, thesignal identifier identifying the target.

The plurality of features may include an amplitude of the syntheticaperture radar signal.

The plurality of features may include a phase of the synthetic apertureradar signal.

The readout neural layer may include a linear classifier.

The input spiking neural network, the multi-layer recurrent neuralnetwork, and the readout neural layer may be implemented by aneuromorphic chip.

The readout neural layer may be configured to compute average spikingrates from the second plurality of spiking signals, wherein the signalidentifier may be computed based on the average spiking rates.

The multi-layer recurrent neural network may include: a first excitatoryneuron layer configured to receive the first plurality of spikingsignals from the input spiking neural network; a first inhibitory neuronlayer; a second excitatory neuron layer; and a second inhibitory neuronlayer connected to the second excitatory neuron layer, the firstexcitatory neuron layer being configured to supply spiking signals to:the first excitatory neuron layer; the first inhibitory neuron layer;and the second excitatory neuron layer, the first inhibitory neuronlayer being configured to supply spiking signals to the first excitatoryneuron layer, the second excitatory neuron layer being configured tosupply spiking signals to: the second excitatory neuron layer; thesecond inhibitory neuron layer; and the readout neural layer, and thesecond inhibitory neuron layer being configured to supply spikingsignals to the second excitatory neuron layer.

The input spiking neural network may include a plurality of inputneurons arranged in a grid, wherein the first excitatory neuron layercomprises a plurality of first excitatory neurons arranged in a grid,the plurality of first excitatory neurons comprising a plurality ofcritical neurons uniformly distributed in the grid, and wherein theplurality of input neurons are connected to the critical neurons tomaintain spatial relationships between the input neurons incorresponding ones of the critical neurons.

The first excitatory neuron layer may include a plurality of excitatoryneurons arranged in a grid, and a neuron of the first excitatory neuronlayer may be configured to supply spiking signals to neurons in a localneighborhood of the grid around the neuron.

The first excitatory neuron layer may include a plurality of firstexcitatory neurons arranged in a grid, the second excitatory neuronlayer may include a plurality of second excitatory neurons arranged in agrid, and the plurality of first excitatory neurons may be connected tothe second excitatory neurons to maintain the spatial relationshipsbetween the first excitatory neurons in corresponding ones of the secondexcitatory neurons.

According to one embodiment of the present invention, a system foridentifying a target in a synthetic aperture radar signal includes:means for extracting a plurality of features from the synthetic apertureradar signal; means for encoding the features as a first plurality ofspiking signals; means for supplying the spiking signals to amulti-layer recurrent neural network to compute a second plurality ofspiking signals; means for computing a signal identifier based on thesecond plurality of spiking signals; and means for outputting the signalidentifier from the readout neural layer, the signal identifieridentifying the target.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, together with the specification, illustrateexemplary embodiments of the present invention, and, together with thedescription, serve to explain the principles of the present invention.

FIG. 1 is a block diagram of a system for identifying or classifying atarget in raw synthetic aperture radar (SAR) data according to oneembodiment of the present invention.

FIG. 2 is a flowchart illustrating a method for identifying orclassifying a target in raw synthetic aperture radar (SAR) dataaccording to one embodiment of the present invention.

FIG. 3 is a block diagram of a feature extractor according to oneembodiment of the present invention configured to compute features fromraw SAR data.

FIG. 4 is a flowchart illustrating a method 210 for computing featuresfrom a segment of the raw SAR data according to one embodiment of thepresent invention.

FIG. 5 is a schematic diagram of a neuromorphic spiking networkaccording to one embodiment of the present invention.

FIG. 6 is a schematic diagram of neurons in a portion of the input layerand their connections to neurons in a corresponding portion of the firstexcitatory neuron layer E1 in accordance with one embodiment of thepresent invention.

FIG. 7 is a schematic diagram of neurons in a portion of the firstexcitatory layer E1 and their connections to neurons in a correspondingportion of the second excitatory neuron layer E2 of the recurrent neuralnetwork in accordance with one embodiment of the present invention.

FIG. 8 presents an example of the local connections in a portion of anexcitatory neuron layer according to one embodiment of the presentinvention.

FIGS. 9A, 9B, and 9C show examples of amplitudes of raw SAR data ofthree civilian vehicles—a Toyota Camry, a Jeep, and a Toyota Tacoma,respectively.

FIGS. 10A, 10B, and 10C, depict examples of SAR feature vectors atdifferent elevations for the three types of vehicles computed inaccordance with an embodiment of the present invention.

FIG. 11 is a block diagram of a computer system that may be used inconjunction with embodiments of the present invention.

DETAILED DESCRIPTION

In the following detailed description, only certain exemplaryembodiments of the present invention are shown and described, by way ofillustration. As those skilled in the art would recognize, the inventionmay be embodied in many different forms and should not be construed asbeing limited to the embodiments set forth herein. Like referencenumerals designate like elements throughout the specification.

Synthetic aperture radar (SAR) data is widely used in both civilian andmilitary applications, such as geographic survey, target recognition,and surveillance. As discussed above, typical SAR based systems performtarget recognition by first generating an image from the received rawSAR data (or phase history data), and then performing target recognitionon the image, such as by displaying the synthesized image to a human.The image synthesis process is generally computationally intensive(e.g., performing a convolution operation over a large amount oftwo-dimensional and/or three-dimensional data), and therefore real-timeSAR image formation requires large size, weight, and power consumption(SWAP) processing hardware which is not suitable for many applicationsand platforms, or which may require offline (e.g., batch, rather thanreal-time, processing). This makes it difficult to apply image formationtechniques in real-time, especially in environments where SWAP areconstrained, such as in an unmanned aerial vehicle (UAV). Furthermore,image synthesis process is a transformation that does not add any newinformation about the targets. Therefore, the raw SAR data contains thesame information as the synthesized SAR image data.

Therefore aspects of embodiments of the present invention are directedto systems and methods for recognizing targets from raw SAR data withoutan intermediate step of synthesizing images from the raw SAR data.Because image synthesis does not add any new information, aspects ofembodiments of the present invention relate to automated targetrecognition systems that do not require an image to be synthesized fromthe raw SAR data. Aspects of embodiments of the present invention enablethe recognition of targets from SAR data in real-time and with reducedpower consumption, which may be particularly useful for applicationswith particular performance requirements and power constraints. In someembodiments of the present invention, at least some aspects areimplemented in a neuromorphic chip, which can operate at highperformance and with low power consumption.

Aspects of embodiments of the present invention relate to a signalrecognition system configured to identify or classify a target found inraw SAR data utilizing a spiking neuromorphic network that generatesspiking sequences based on features computed from complex raw SAR data,such as phase and amplitude. For example, the system may be configuredto identify whether the raw SAR data was reflected off a particular typeof aircraft (e.g., distinguishing between specific models of aircraft),or a particular model of a car (e.g., a Toyota Camry versus a HondaAccord versus a Jeep). A system for using a spiking neuromorphic networkfor analyzing raw SAR data is described in U.S. patent application Ser.No. 15/784,841, “System and Method for Synthetic Aperture Radar TargetRecognition Utilizing Spiking Neuromorphic Networks” filed in the UnitedStates Patent and Trademark Office on Oct. 16, 2017, the entiredisclosure of which is incorporated by reference herein.

Neuromorphic computing is a biologically-inspired approach based onobservations that the human brain is able to compute complex functionswhile utilizing a very small amount of power. For example, the humanbrain has excellent capabilities in object recognition. In someneuromorphic computing devices, the physical hardware of the deviceimplements electronic “neurons” that communicate with one anotherthrough spiking sequences (e.g., voltage pulses), in a manner thatresembles the spiking sequences of signals between neurons in a humanbrain.

As such, one approach to applying neuromorphic computing to objectlearning and recognition is to encode features of the input data in thespiking signal domain. A binary spiking neural network may becomputationally more efficient (e.g., energy efficient) than areal-valued neural network because processing spiking signals (binarysequences) consumes much less power than processing real-valued signals.Therefore, in some embodiments of the present invention, recognizing SARtargets utilizing a spiking neural network will greatly improveefficiency of power consumption and accuracy of target recognition thana comparable real-valued neural network.

Aspects of embodiments of the present invention relate to a targetrecognition system that is configured to recognize different targetsfrom raw SAR data (e.g., phase history data) using a multi-layerrecurrent spiking neural network for capturing differentspatial-temporal patterns in the raw SAR data. In some embodiments, theneural network includes local recurrent neural connections, which areable to capture at least some spatial-temporal correlations and patternsin the data.

Some aspects of embodiments of the present invention relate to the useof spike-time-dependent plasticity (STDP) to perform unsupervisedtraining on the multi-layer recurrent spiking neural network, where theunsupervised training process causes the neural network to self-organizeto perform the target recognition function based on raw SAR feature datawithout forming SAR images, as described in more detail below.

FIG. 1 is a block diagram of a system for identifying or classifying atarget in raw synthetic aperture radar (SAR) data according to oneembodiment of the present invention. As shown in FIG. 1, in oneembodiment, a system 100 for identifying or classifying a targetincludes: a feature extractor 110 that is configured to extract featuresfrom raw SAR data; an input spiking neural network 130 configured toencode the extracted SAR features as spiking sequences; a multi-layerrecurrent neural network 150; and a readout neural layer 170, whichincludes a classifier (e.g., a linear classifier trained using asupervised training method such as a rank-1 learning technique) that isconfigured to classify different SAR targets from spiking sequencesgenerated from the SAR features, and to output an identification of thetarget (a “target identifier”). The readout layer may be trained by asupervised learning technique. Experimental results show thatimplementations of embodiments of the present invention are effective inclassifying targets from raw SAR data.

As such, aspects of embodiments of the present invention relate toclassifying SAR targets utilizing a spiking neural network that is veryefficient in power consumption; and/or classifying SAR targets withoutforming SAR images.

FIG. 2 is a flowchart illustrating a method 200 for identifying orclassifying a target in raw synthetic aperture radar (SAR) dataaccording to one embodiment of the present invention.

Synthetic Aperture Radar (SAR) Features

One commonly used data format for raw synthetic aperture radar (SAR)data is phase history data, represented as a two-dimensional (2D) arrayof complex numbers of reflected RF signals from the targets. Onedimension of the array represents frequency bins while the otherdimension represents the number of received radar pulses. Because a SARsensor generally covers a large area, raw SAR data is typically a verylarge 2D array of complex numbers.

According to one embodiment, in operation 210, the feature extractor 110receives synthetic aperture radar (SAR) raw data (e.g., from a radarantenna), and computes features from the SAR raw data. The 2D complexphase history raw SAR data does not have an intuitive or natural mappingto spiking signals. In some embodiments of the present invention, fourdifferent features are computed from raw SAR data: amplitude,off-amplitude, positive phase, and negative phase.

FIG. 3 is a block diagram of a feature extractor 110 according to oneembodiment of the present invention configured to compute features fromraw SAR data.

FIG. 4 is a flowchart illustrating a method 210 for computing featuresfrom a segment of the raw SAR data according to one embodiment of thepresent invention.

According to some embodiments of the present invention, the featureextractor 110 divides the 2D array of raw SAR feature data into asequence of subsets (or time windows) of the 2D array. Using a sequenceof 2D subsets (or time windows) to represent the 2D SAR array enablesembodiments of the present invention to: capture some spatialrelationships and/or patterns presented in raw SAR data; utilize thememory property of recurrent spiking neural networks to capture thetemporal correlations presented in the sequence of 2D subsets; processthe raw SAR data in real-time or substantially real-time because, inpractice, raw SAR data is generally obtained as a sequence of 2D RFsignals.

Given that the raw SAR data is a matrix or two-dimensional (2D) array ofcomplex values, the data point of the SAR data s(i,j) at position (i,j)in the two-dimensional array is given by s(i,j)=a(i,j)+jb(i,j) (wherethe coefficient j refers to the unit imaginary number j=√{square rootover (−1)}, the parameters i and j are indices into the two-dimensionalarray, and a(i,j) and b (i,j) respectively refer to the real andimaginary components of the data point at s(i,j)). In operation 212, theamplitude extractor 112 extracts the amplitude feature Am(i,j) for eachdata point s(i,j) (e.g., for each coordinate pair (i,j)) in accordancewith Equation 1:Am(i,j)=√{square root over (a(i,j)² +b(i,j)²)}The off-amplitude extractor 114 extracts off-amplitude feature inaccordance with Equation 2:Am _(off)(i,j)=Am _(max) −Am(i,j)

The phase extractor 118 computes the phase values Ph(i,j) in operation216 for each data point s(i,j)=a(i,j)+jb(i,j) in accordance withEquation 3:

${{Ph}\left( {i,j} \right)} = {\tan^{- 1}\left( \frac{b\left( {i,j} \right)}{a\left( {i,j} \right)} \right)}$

The raw SAR feature point is given in Equation 4 as:

${{Ft}\left( {i,j} \right)} = \begin{bmatrix}{{Am}\left( {i,j} \right)} \\{{Am}_{off}\left( {i,j} \right)} \\{{Ph}\left( {i,j} \right)}\end{bmatrix}$

Because the phase features Ph(i,j) have a range of [0.0, π], in oneembodiment, in operation 214, the normalizer 116 normalizes theamplitude features Am(i,j) to the same dynamic range of [0.0, π]. Assuch, the normalized amplitude features Am(i,j) are given by Equation 5:

${\overset{\_}{Am}\left( {i,j} \right)} = {\pi*\frac{{Am}\left( {i,j} \right)}{{Am}_{m\;{ax}}}}$such that the normalized raw SAR feature point is given by Equation 6as:

$\quad\begin{bmatrix}{\overset{\_}{Am}\left( {i,j} \right)} \\{{\overset{\_}{Am}}_{off}\left( {i,j} \right)} \\{{Ph}\left( {i,j} \right)}\end{bmatrix}$

Because the raw SAR data is a 2D complex array, the feature vectors canbe arranged based on row data points or column data points to generate asequence of feature vectors for the raw data array. Defining F_(Am) tobe a 2D vector of size M×N (M rows by N columns) of the amplitudefeatures Am(i,j) (or the normalized features Am _(ij)) and definingF_(Ph) to be a 2D vector of size M×N of the phase features Ph, a 2D SARfeature vector F_(SAR) of size 2M×N is constructed in operation 220 asshown in Equation 7:

$F_{SAR} = \begin{bmatrix}F_{Am} \\F_{Ph}\end{bmatrix}$

Both the F_(Am) and F_(Ph) are subsets of the original 2D amplitude andphase arrays computed from the raw SAR data. A raw SAR data set isrepresented by a sequence of 2D SAR feature vectors.

Spiking Neural Networks

Recurrent neural networks have the capability to capture thespatial-temporal correlations and/or patterns in the input data, andmulti-layer recurrent neural networks have the potential to representthe spatial-temporal correlations/patterns of the input data indifferent abstraction levels, which is a desired property forclassifying complicated targets.

In some embodiments of the present invention a neuromorphic spikingnetwork is implemented by a general purpose processor (CPU) 140 or ageneral purpose graphics processing unit (GPU) 140 configured tosimulate the input layer 130, the multi-layer recurrent neural network150 (including the first excitatory neuron layer E1 152, the firstinhibitory neuron layer I1 154, the second excitatory neuron layer E2156, and the second inhibitory neuron layer I2 158), the readout layer170, and the connections within and between the layers. In someembodiments, the neuromorphic spiking network is implemented by aneuromorphic chip such as a neuromorphic chip produced by HRLLaboratories of Malibu, Calif., or the TrueNorth neural net chipproduced by International Business Machines Corporation (IBM) of Armonk,N.Y. In circumstances of limited size, weight, and power (SWAP), aneuromorphic chip may provide reduced SWAP in comparison to a generalpurpose processor. In some embodiments of the present invention, such asin a production device, where a set of weights W and/or other parametersof the neural network has already been determined through the trainingprocess, the neural network can be implemented utilizing a dedicatedcircuit based on the fixed parameters. The dedicated circuit may be, forexample, a field programmable gate array (FPGA) or an applicationspecific integrated circuit (ASIC).

FIG. 5 is a schematic diagram of a neuromorphic spiking networkaccording to one embodiment of the present invention. As shown in FIG.5, the neuromorphic spiking network includes the input layer 130, whichencodes received SAR feature vectors as input spiking signals. The inputspiking signals are supplied through connections 132 to a multi-layerrecurrent neural network 150, which generates a plurality of outputspiking sequences. The readout layer 170 computes a target identifier(or target ID) based on the output spiking sequences received throughconnections 172 from the multi-layer recurrent neural network 150, wherethe target identifier identifies a target that was detected in the rawSAR data that was supplied as input to the system.

In one embodiment, in operation 230, each value of a real-valued featurevector is converted into a corresponding spiking sequence generated by acorresponding input neuron of the input spiking neural network 130. Assuch, according to one embodiment, the size of the input spiking neuralnetwork 130 (e.g., the number of neurons in the input spiking neuralnetwork 130) is equal to the number of features in the input 2D featurevector.

Each spiking sequence generated by each input neuron is made up ofspikes that are randomly (or pseudo-randomly) generated in accordancewith a Poisson distribution or exponential distribution having a meanspiking rate that corresponds to the value of its corresponding feature(e.g., a value in the normalized range of [0, π]). The exponentialdistribution may be used as a computationally simpler approximation ofthe Poisson distribution. In particular, the real values of the featurevector are treated as average (e.g., mean) values of the distribution,and a random number generator is used to produce spikes such that therandomly generated spiking sequences have mean values corresponding tothe values of the feature vector. For example, if each feature vectorF_(SAR), included 2M×N different features, those features would beconverted into 2M×N different spiking sequences, each spiking sequencehaving a spiking rate distribution in accordance with the real value ofthe corresponding feature.

More precisely, according to one embodiment, if F_(SAR)(i,j) is the realfeature value (computed from the raw SAR data in operation 210) suppliedas input to a neuron at position (i,j) in the input layer 130, inoperation 230, a random number k is generated (e.g., by a random numbergenerator) for a simulation time point t, where k is generated with theexponential distribution function of Equation 8:

${f\left( {t,\lambda} \right)} = \left\{ \begin{matrix}{{\lambda\; e^{{- \lambda}\; t}},} & {t \geq 0} \\{0,} & {t < 0}\end{matrix} \right.$where λ=F_(SAR)(i,j). The input layer neuron at position (i,j) of theinput layer 130 generates a sequence of spikes, where the time intervalbetween spikes is equal to the number k. When the input feature value λchanges, a new time interval k is generated in accordance with theexponential function ƒ(t,λ).

Because the values of the feature vector generally change over time(e.g., the values may differ from one vector (row vector or columnvector) of the SAR data to the next), the generated spiking sequencesmay have varying mean values over time.

As shown in FIG. 5, in one embodiment of the present invention, themulti-layer recurrent neural network 150 includes a first excitatoryneuron layer E1 152, a first inhibitory neuron layer I1 154, a secondexcitatory neuron layer E2 156, and a second inhibitory neuron layer I2158. The multi-layer recurrent neural network 150 is configured, in partthrough an unsupervised training process, to categorize and clustersimilar information from the feature vectors representing raw SAR data.The resulting clustered spikes are supplied to the readout layer tocompute a classification of the target based on the output of themulti-layer recurrent neural network 150.

In the multi-layer recurrent neural network 150 shown in FIG. 5, eachneuron receives synaptic signals or “spikes” from other neurons andreleases synaptic signals (other “spikes”) to other neurons. In someembodiments, each neuron is modeled by the integrate-and-fire model,where a neuron is modeled as a capacitor C in parallel with a resistor Rdriven by a current I(t). In a leaky integrate-and-fire model, theneuron firing (e.g., emission of a spike) is described by the dynamicequation of a neuron membrane potential and current, as given byEquation 9, below:

${\tau_{m}\frac{{du}(t)}{dt}} = {{- {u(t)}} + {{RI}(t)}}$where u(t) is the membrane potential (e.g., the voltage across thecapacitor C), I(t) is the membrane current, and the constants τ_(m) andR are the membrane time constant and resistance of the neuron,respectively. When the membrane potential u(t) is greater than or equalto a firing threshold V_(th) at time t_(f), the neuron outputs a spikewhich is scaled by the connection weight wδ(t−t_(f)) (in the case of aninhibitory neuron, the spike has the opposite sign from the excitatoryneuron, e.g., −wδ(t−t_(f))), and, after firing, the capacitor C is reset(e.g., set to zero volts by shorting the capacitor to ground). Thefiring thresholds V_(th) for the excitatory neurons and the inhibitoryneurons are different and determined by a parameter search procedure, asdescribed in U.S. patent application Ser. No. 15/784,841, “System AndMethod for Synthetic Aperture Radar Target Recognition Utilizing SpikingNeuromorphic Networks,” the entire disclosure of which is incorporatedby reference herein. According to one embodiment, the length of thespike δ is equal to one time step (e.g., one sample).

Qualitatively, as input signals (e.g., spikes or the input values fromthe feature vector) arrive at the input of the neuron, the spikesaccumulate over time in a capacitor C. The “leakiness” of the neuroncauses the accumulated voltage at the capacitor C to decrease over time.However, if sufficient spiking energy arrives at the neuron (e.g.,spikes can arrive at various rates and with various voltages and/orwidths), then the voltage at the capacitor C may eventually exceed thethreshold voltage V_(th), at which point the neuron fires by emitting aspike on its output, and by resetting the capacitor to an initialvoltage.

FIG. 6 is a schematic diagram of neurons in a portion of the inputneural layer 130 and their connections to neurons in a correspondingportion of the first excitatory neuron layer E1 152 of the multi-layerrecurrent neural network 150 in accordance with one embodiment of thepresent invention. In some embodiments, the first excitatory neuronlayer E1 152 is larger than the input layer 130, and the connections 132between the input layer 130 and the first excitatory neuron layer E1 152are on a one-to-one basis (e.g., one neuron of the input layer 130 toone corresponding “critical” neuron of the first excitatory neuron layerE1 152) with fixed synaptic weights (e.g., in some embodiments, there isno need to learn or modify these weights because the function ofconnections 132 is to pass the spikes generated by the input layer 130to the multi-layer recurrent neural network 150). As shown in FIG. 6,each neuron of the input layer 130 and the first excitatory layer E1 152is associated with a position (e.g., in a square grid) within itsrespective excitatory layer. In one embodiment, the critical neurons onthe first excitatory neuron layer E1 152 are uniformly distributed(e.g., they are located on a uniform sub-sampling grid on the firstexcitatory neuron layer E1 152).

For example, the six neurons of the portion of the input layer 130depicted in FIG. 6 supply input spiking signals (corresponding to thereal values of the features) through connections 132 to corresponding“critical” neurons (shown in FIG. 6 as filled-in circles) of the firstexcitatory neuron layer E1 152. As shown in FIG. 6, the critical neuronsare spaced apart and appear in every other row and in every other columnof the grid of the first excitatory neuron layer E1 152. In addition, asseen in FIG. 6, the spatial relationships between adjacent neurons ofthe input layer 130 are maintained or preserved in the correspondingcritical neurons on the first excitatory neuron layer E1 152. Forexample, input layer neuron 130A in the upper left portion of the inputlayer 130 is connected to critical neuron 152A in the upper left portionof the first excitatory neuron layer E1 152. Input neuron 130A islocated immediately above input neuron 130B and immediately to the leftof input neuron 130D. Likewise, critical neuron 152A is the nextcritical neuron above critical neuron 1526 and is the next criticalneuron to the left of critical neuron 152D. As another example, inputneuron 130B is between input neurons 130A and 130C and, likewise,corresponding critical neuron 1526 is between critical neurons 152A and152C. Similarly, input neuron 130E is to the right of input neuron 1306and between input neuron 130D and input neuron 130F, and likewisecritical neuron 152E is to the right of critical neuron 1526 and betweencritical neuron 152D and critical neuron 152F.

FIG. 7 is a schematic diagram of neurons in a portion of the firstexcitatory layer E1 152 and their connections 155 to neurons in acorresponding portion of the second excitatory neuron layer E2 156 ofthe multi-layer recurrent neural network 150 in accordance with oneembodiment of the present invention. In some embodiments, the secondexcitatory neuron layer E2 156 is smaller than the first excitatoryneuron layer E1 152. The connections 155 from the first excitatoryneuron layer E1 152 to the second excitatory neuron layer E2 156 form a“many-to-one” structure where multiple adjacent neurons from the firstexcitatory neuron layer E1 152 are connected to a single neuron in thesecond excitatory neuron layer E2 156. In some embodiments, theconnections 155 are fixed weight connections (e.g., not learned during atraining process). For example, as shown in FIG. 7, in one embodiment,groups of four adjacent neurons of the first excitatory neuron layer E1152 are connected to (e.g., supply their outputs to) a singlecorresponding neuron of the second excitatory neuron layer E2 156. Morespecifically, in the example shown in FIG. 7, the four neurons 152-1 ofthe first excitatory neuron layer E1 are connected to a single neuron156-1 of the second excitatory neuron layer E2, the four neurons 152-2of the first excitatory neuron layer E1 are connected to a single neuron156-2 of the second excitatory neuron layer E2, the four neurons 152-3of the first excitatory neuron layer E1 are connected to a single neuron156-3 of the second excitatory neuron layer E2, and the four neurons152-4 of the first excitatory neuron layer E1 are connected to a singleneuron 156-4 of the second excitatory neuron layer E2. However,embodiments of the present invention are not limited thereto and mayinclude larger or differently sized groups of adjacent neurons (e.g., a3×3 group, a 3×2 group, or a 1×4 group of neurons from first excitatoryneuron layer E1 152 may be connected to a single neuron of the secondexcitatory neuron layer E2 156). In addition, in some embodiments,adjacent neurons in the second excitatory neuron layer E2 156 mayreceive inputs from overlapping adjacent groups of neurons of the firstexcitatory neuron layer E1 152.

The connections 155 between the first excitatory neuron layer E1 152 andthe second excitatory neuron layer E2 156 maintain the spatialrelationship of the information from the neurons of the first excitatoryneuron layer E1 152 in the neurons of the second excitatory neuron layerE2 156. For example, in the first excitatory neuron layer E1 152, thefirst group of four neurons 152-1 of the first excitatory neuron layerE1 are above the second group of four neurons 152-2 and to the left ofthe third group of neurons 152-3. Likewise, in the second excitatoryneuron layer E2 156, the first neuron 156-1 is above the second neuron156-2 and to the left of the third neuron 156-3, where these neurons ofthe second excitatory neuron layer E2 156 are respectively connected tothe first group of four neurons 152-1, the second group of four neurons152-2, and the third group of four neurons 152-3 of the first excitatoryneuron layer 152.

Accordingly, the second excitatory neuron layer E2 156 can be thought ofas a summary of a group of signals (e.g., summary of the outputs ofadjacent neurons) of the first excitatory neuron layer E1 152, and thespiking sequences may be closely correlated to each other.

The neurons within the first excitatory neuron layer 152 are alsoconnected to one another over local connections 152R. Likewise, theneurons within the second excitatory neuron layer 156 are connected toone another over local connections 156R. FIG. 8 is a schematic diagramof a portion of the first excitatory neuron layer E1 152 according toone embodiment of the present invention.

According to one embodiment, excitatory neurons are connected to otherexcitatory neurons in both the first excitatory neuron layer E1 152 andthe second excitatory neuron layer E2 156 layer by local neighborhoodconnections (e.g., connections to other neurons within their ownlayers), and their synaptic weights are determined byspike-time-dependent plasticity (STDP) learning, as described in moredetail below. FIG. 8 presents an example of the local connections 152Rin a portion of the first excitatory neuron layer E1 152 according toone embodiment of the present invention. In some embodiments of thepresent invention, similar local connections 156R connect the neurons ofthe second excitatory neuron layer E2 156.

As shown in FIG. 8, each neuron (e.g., the neuron shown in black) isbi-directionally connected to each of the eight adjacent neurons. Inother words, each neuron is configured to transmit spiking signals toeach of the eight adjacent neurons (upper left, upper center, upperright, right, lower right, lower center, lower left, and left). Thelocal neighborhood connections 152R and 156R form the recurrentcomputing structure of the multi-layer recurrent neural network 150,where various portions of the network 150 receive signals originatingfrom previous time points (e.g., spiking signals computed from earlierreceived raw SAR data) and combine these signals with current spikingsignals.

Referring back to FIG. 5, the multi-layer recurrent neural network 150also includes a first inhibitory neuron layer I1 154 connected to thefirst excitatory neuron layer E1 152 through connections 153 and asecond inhibitory neuron layer I2 158 connected to the second excitatoryneuron layer E2 156 through connections 157. The first inhibitory neuronlayer I1 154 and the second inhibitory neuron layer I2 158 function tostabilize the neuron firing rates within the multi-layer recurrentneural network 150.

The connections 153 include a first set of connections from neurons inthe first excitatory neuron layer 152 to neurons in the first inhibitoryneuron layer 154, where the number of connections is set based on aconnection ratio parameter (a connection probability) (described in moredetail below), where the connections are selected based on a uniformlyrandom distribution (e.g., each neuron in the first excitatory neuronlayer 152 is connected to a set of neurons uniformly randomly selectedfrom the first inhibitory neuron layer 154). Similarly, the connections153 further include a second set of connections from neurons in thefirst inhibitory neuron layer 154 to the first excitatory neuron layer152, where the number of connections is set based on another connectionratio parameter, where the connections are selected based on a uniformlyrandom distribution. The synaptic weights of these random connections153 are set (or learned) using spike-time-dependent plasticity (STDP)learning, as described in more detail below.

The connections 157 between the second excitatory neuron layer 156 andthe second inhibitory neuron layer 158 are substantially similar to theconnections 153 between the first excitatory neuron layer 152 and thefirst inhibitory neuron layer 154, and the number of connections may becontrolled by a corresponding set of connection ratio parameters. Thesynaptic weights of these connections 157 are also set (or learned)using spike-time-dependent plasticity (STDP) learning, as described inmore detail below.

According to some embodiments of the present invention, the connections172 between the second excitatory neuron layer E2 156 and the readoutlayer 170 is a full-connection structure, in which every neuron in thesecond excitatory neuron layer E2 156 is connected to every neuron inthe readout layer 170. In some embodiments, the synaptic weights of theconnections 172 are determined through a supervised learning method,where the readout layer 170 is configured to operate as a linearclassifier.

Network Training using Unsupervised Learning

Some aspects of embodiments of the present invention relate to trainingor determining the weights of the connections 152R, 153, 156R, and 157in an unsupervised manner, in order to prepare the multi-layer recurrentneural network 150 for the task of target classification. In someaspects of embodiments of the present invention, spike-time-dependentplasticity (STDP) learning is used to organize the multi-layer recurrentneural network 150 into hierarchical clusters that may facilitate thetarget classification process.

During STDP learning, if t_(pre) and t_(post) are, respectively, thespiking times for a pre-synaptic spike and a post-synaptic spike, thecorresponding synaptic weight (“synaptic conductance”) is computed withrespect to Equations (10), (11), and (12) below:g _(new) =g _(old) +ΔgΔg=g ⁻max*F(Δt)

${F\left( {\Delta t} \right)} = \left\{ \begin{matrix}{A_{+}*{\exp\left( \frac{\Delta t}{\tau_{+}} \right)}} & {{{if}{\mspace{14mu}\ }\Delta\; t} < 0} \\{{- A_{-}}*{\exp\left( \frac{\Delta t}{\tau_{-}} \right)}} & {{{if}{\mspace{14mu}\ }\Delta\; t} \geq 0}\end{matrix} \right.$where Δt=t_(pre)−t_(post). The constants A₊ and A⁻ determine the maximumamount of synaptic modification. The time constraints τ₊ and τ⁻determine the ranges of pre- to post-synaptic spike intervals.Qualitatively, the STDP learning rule is that, if a pre-synaptic spikecan generate a post-synaptic spike immediately, the synaptic weight isincreased; otherwise, it is decreased. As a result, a high value in asynaptic weight refers to that the two neurons connected by the synapticweight are closely coupled and are acting together. On the other hand, asmall value in synaptic weight means that the activity of the twoneurons have no impact on each other.

In some embodiments, to improve the efficiency of neurons, the synapticweights of the multi-layer recurrent network are normalized after agiven time period during the weight training process. More specifically,in some embodiments, for a given neuron (m, n), the function g(i,j,t)represents the synaptic weight of the connection from neuron (i,j) toneuron e(m,n) at time t. During an unsupervised STDP learning process,after a given learning time period (or normalization period) T_(n)(e.g., the time between t+T_(n) and t+T_(n)+1), the synaptic weights gare normalized to a set or predefined constant C in accordance withEquation 13:

${g\left( {i,j,{t + 1}} \right)} = \left\{ \begin{matrix}{\frac{C*{g\left( {i,j,t} \right)}}{\sum\limits_{u,v}{g\left( {u,v,t} \right)}},} & {{t = {kT}_{n}},{k = 1},2,3,\ldots} \\{{g\left( {i,j,{t + 1}} \right)}\ ,} & {otherwise}\end{matrix} \right.$k=1, 2, 3, . . . .In some embodiments, the normalization period τ_(n) is in the range of50 ms to 100 ms.

Applying weight normalization in accordance with some embodiments of thepresent invention can mitigate or prevent quick weight saturation duringthe unsupervised learning process and thereby improve the utilization ofevery neuron in the network. This is because, if the synaptic weights ofa neuron become saturated, that neuron is no longer useful in thelearning process.

Accordingly, the unsupervised STDP learning process adjusts the weightsw of the connections between neurons in the multi-layer recurrent neuralnetwork 150 to organize the neurons to cluster their spiking based onthe input features (where the input features were computed directly fromthe raw SAR data). As noted above, the output of the multi-layerrecurrent neural network 150 is supplied to a readout layer 170 throughconnections 172.

In operation 250, the features of the raw SAR data, encoded as spikingsignals, are supplied to the multi-layer recurrent neural network 150 togenerate another set of spiking signals, which are supplied in operation270 to the readout layer 170 to compute a classification (e.g.,identifying the type of vehicle appearing in the SAR data).

Readout Layer Training Using Supervised Learning

Some aspects of embodiments of the present invention relate to a readoutlayer 170, which plays the role of linear classification based onaveraged spiking rates from the second excitatory neuron layer E2 156.For a given time period T (a presentation time of an input 2D featurevector to the input layer 130), the resulting average spiking rate ofthe second excitatory neuron layer E2 156 can be calculated inaccordance with Equation 14:

${r_{avg}\left( {m,n} \right)} = {\frac{1}{T}{\sum\limits_{t}^{T}{{spk}\left( {m,n,t} \right)}}}$where spk(m, n, t) is the spike generated by the neuron e(m, n) at timet. In some embodiments, the presentation time T is in the range of 200ms to 500 ms.

The readout neural layer 150 maps the average spiking rates r_(avg)(m,n)of the neurons of the second excitatory neuron layer E2 156 to a targetclass c_(i) based on a plurality of classifier weights (e.g., theweights of the connections 172) arranged in a mapping weight matrix W inaccordance with Equation 15:c _(i) =Wr _(i) ,i=1,2,3, . . . ,Mwhere r_(i) is a vector containing all averaged firing rates from allneurons in the output layer at time i, where M is the total number oftime indices, where c_(i) refers to a target class computed at time i.In other words, Equation 15 indicates that, at time index i, a vector ofaverage firing rates r_(i) from the second excitatory neuron layer E2156 can be used to compute a target class c_(i) at time i among Kdifferent classes. For example, when the network is configured todistinguish between three different types of vehicles (e.g., a ToyotaCamry versus a Honda Accord versus a Jeep), then K=3 and each differenttype of vehicle corresponds to a different class c.

Some aspects of embodiments of the present invention relate to trainingor learning the synaptic weights (represented by matrix W) of theconnections 172 from the second excitatory neuron layer E2 156 to thereadout layer 170 using a supervised learning method (e.g., usinglabeled training data). Some embodiments of the present invention relateto the use of a Rank-1 learning rule to train these weights, where theRank-1 learning rule is similar to a mean-squared learning rule.

Generally, the Rank-1 learning rule maps the training data into asubspace such that all of the common components in the training data areremoved before utilizing the training data to train a classifier, wherethe mapping rule is generally learned from the training data. Thetraining data, in this case, includes raw SAR data that are labeledbased on the classifications such containing reflected signals fromvarious types of vehicles (e.g., different models of cars and/oraircraft) and the like. The removal of the common components improvesthe efficiency of the training process, as components that do notprovide information that helps in distinguishing between the variousclasses c_(k) of signals need not be processed.

To remove the common components in the training vectors, the trainingvectors are mapped into a subspace by a signal sub-space mapping matrixA as shown in Equation 16:

$A = {A - {k_{i}\frac{k_{i}^{T}}{\alpha_{i}}}}$where:k _(i) =Ar _(i)α_(i)=1+k _(i) r _(i)where the signal sub-space mapping matrix A is learned from the trainingdata during the training, where k_(i) is a subspace vector computed fromr_(i), which is the i-th average spiking rate vector, as shown above inEquation 16 and α_(i) is a normalization constant.

Accordingly, the weight matrix W, which is the synaptic weights of theoutputs of neuron e(m, n) of the second excitatory neuron layer E2 156to the readout layer 170, can be updated in an iterative supervisedlearning process in accordance with Equation 17:

$W = {W - {E_{i}\frac{k_{i}^{T}}{\alpha_{i}}}}$where the error signal E_(i) is:E _(i) =c _(i) −t _(i)where c_(i) is the computed output of readout network 170 based on thecurrent synaptic weights W, and t_(i) is the target label for thetraining.

As a result, in operation 290, the readout neural layer 150 outputs asignal identifier or target identifier c_(i) that identifies theclassification of the target found in the raw SAR data. For example, inan embodiment directed to reconnaissance, the signal identifier mayidentify the classification of the target in the SAR data as a car, atruck, or a tank.

Accordingly, some embodiments of the present invention are directed to aneural network, including a multi-layer recurrent neural network that isconfigured to detect the presence of targets within raw SAR data,classify the detected targets, and to output the resultingclassifications.

In some embodiments, one or more of the input neural network 130, themulti-layer recurrent neural network 150, and the readout neural layer150 may be implemented in an integrated device, such as an integratedcircuit. In some embodiments, all of the components of the system 100may be implemented utilizing an electronic circuit fabricated on asingle substrate, such as a mixed signal chip that includes one or moredigital signal processors (DSPs) and an analog neuromorphic circuit. Thevalues of the mapping weight matrix W may be fixed in the neuralnetwork, such as by being stored in read-only memory of the integratedcircuit or physically implemented in spike generating circuit of theintegrated circuit (e.g., if a spike generating circuit is controlledbased on a resistance value). Similarly, the values of the weights ofthe synaptic connections within the multi-layer recurrent neural network150, such as connections 152R within the first excitatory neuron layerE1 152, the values of the weights of the connections 156R within thesecond excitatory neuron layer E2 156, the values of the weights of theconnections between the first excitatory neuron layer E1 152 and thefirst inhibitory neuron layer 11 154, and the values of the weights ofthe connections between the second excitatory neuron layer E2 156 andthe second inhibitory neuron layer I2 158, may also be fixed in theneural network.

Experimental Results

One embodiment of the present invention was implemented to test theperformance of the system utilizing data captured from simulated SARdata. The testing data included simulated X-band SAR phase history ofcivilian vehicles. The data for each vehicle included three sets of rawSAR data collected from different elevation angles (e.g., from an aerialvehicle with a downward pointing SAR system) of 40 degree, 50 degree,and 60 degree. In this test, the raw SAR data of three vehicles—a ToyotaCamry, a Jeep, and Toyota Tacoma—were used. FIGS. 9A, 9B, and 9C showexamples of amplitudes of raw SAR data of the Toyota Camry, the Jeep,and the Toyota Tacoma respectively.

According to an experiment performed in accordance with one embodimentof the present invention, the original size of the testing data was 512data points (frequency bins)×5600 data points (number of pulses). Thenumber of pulses was down-sampled to 256 data points and the frequencybins was divided by 8 to form the 2D feature vectors. Accordingly, inthe experiment, the 2D feature vectors had a size of 8×512 (8×256 foramplitude and 8×256 for phase). In our experiments, we used a sequenceof sixty 2D feature vectors to represent a raw SAR data set; eachfeature sequence (data set) was presented to the network twenty times.In total, for each vehicle, one thousand two hundred 2D vectors weregenerated for training and testing. FIGS. 10A, 10B, and 10C, depictexamples of SAR feature vectors at different elevations for the threetypes of vehicles computed in accordance with an embodiment of thepresent invention.

The neural network in accordance with one embodiment of the presentinvention, as used in the experiment, includes an input layer 130 having8×512 neurons; a first excitatory neuron layer E1 152 having 16×1024neurons; a second excitatory neuron layer E2 156 having 4×256 neurons; afirst inhibitory neuron layer I1 154 having 4×128 neurons; a secondinhibitory neuron layer I2 158 having 4×32 neurons; and a readout layer170 having 5 neurons. A local neighborhood size of 5×5 neurons was usedfor the recurrent local connections 152R in the first excitatory neuronlayer E1 152, and a local neighborhood size of 3×3 neurons was used forthe recurrent local connections 156R in the second excitatory neuronlayer E2 156. According to one aspect of embodiments of the presentinvention, the size of the local neighborhood connections is determinedby the range of spatial-temporal correlations in the data. Because theE2 Layer is an abstraction or summary of the E1 Layer, the size of localneighborhood in the E2 Layer is generally smaller than the localneighborhood for the E1 Layer. In one embodiment, the synaptic weightsfrom the input layer 130 to the first excitatory neuron layer E1 152 andfrom the first excitatory neuron layer E1 152 to the second excitatoryneuron layer E2 156 are fixed (e.g., with a value of 32). The randomconnections between the first excitatory neuron layer E1 152 and thefirst inhibitory neuron layer I1 154 and between the second excitatoryneuron layer E2 156 and the second inhibitory neuron layer I2 158 wereset to 0.005 (0.5%). The system parameters for STDP leaning aresummarized in Table 1.

TABLE 1 Parameter Value A₊ 10.0 A⁻ 10.0 τ₊ 194.0 τ⁻ 227.0 KDECAY 204.0 Eto I connection ratio 0.005 E V_(th) 279.0 I V_(th) 376.0 I to Econnection ratio 0.005

The parameters A₊, A⁻, τ₊, and τ⁻ are used in Equation 12, above. Theparameter E V_(th) is the neural firing threshold for excitatory neurons(neurons in excitatory layers 152 and 156) and I Vth is the neuralfiring threshold for inhibitory neurons (neurons in inhibitory layers154 and 158). The E to I connection ratio and the I to E connectionratio are the random connection rates between for the connections 153between the first excitatory neuron layer E1 152 and the firstinhibitory neuron layer I1 154 and for the connections 157 between thesecond excitatory neuron layer E2 154 and the second inhibitory neuronlayer I2 158.

In the experiment, the raw SAR data representing elevation angles of 40degree and 60 degree were used to train the network (both in theunsupervised training of the multi-layer recurrent neural network 150and supervised training of the readout layer 170). The raw SAR data ofelevation angle 50 degrees was used for testing. Each data set wasrepresented by a sequence of sixty 2D feature vectors (examples of thesefeature vectors are shown in FIGS. 9A, 9B, and 9C), where each 2Dfeature vector was presented to the network for 100 milliseconds. Thetime period of synaptic weight normalization was set to 50 milliseconds.The input neural firing range was set to a range of 5.0 Hz to 100.0 Hz.In various embodiments of the present invention, the connection ratiobetween excitatory neurons and inhibitory neurons (between E1 and I1;between E2 and I2) can be adjusted to control neural firing rates of thenetwork. In experiments, the network had average firing rates of 8.9 Hzfor excitatory neurons and 22.7 Hz for inhibitory neurons, which are ina reasonable range because the spiking firing rates for human brain aregenerally below 20 Hz. The classification results of the experiment aresummarized in Table 2. The trained system produced an averageclassification rate of 92.13%.

TABLE 2 Computed ID True ID Camry Jeep Tacoma Camry 0.9892 0.0083 0.0025 Jeep 0.0508 0.9417  0.0075 Tacoma 0.0400 0.1217  0.8383 AverageClassification Rate 92.31%

As seen in Table 2, the system correctly classified the Camry data as aCamry 98.92% of the time (incorrectly classifying the Camry as Jeep0.83% of the time and incorrectly classifying the Camry as Tacoma 0.25%of the time), correctly classified the Jeep as a Jeep 94.17% of the time(incorrectly classifying the Jeep as a Camry 5.08% of the time andincorrectly classifying the Jeep as a Tacoma 0.75% of the time, andcorrectly classifying the Tacoma data as a Tacoma 83.83% of the time(incorrectly classifying the Tacoma as a Camry 4.00% of the time andincorrectly classifying the Tacoma as a Jeep 12.17% of the time).

As such, the experimental test produced an accuracy of about 92.31% inaverage classification rate, even without forming SAR images, andthrough the use of a multi-layer recurrent spiking neural network, bothcharacteristics of which result in less energy consumption thancomparative neural networks that process SAR images, due in part to thereduced computational requirements for SAR data based targetrecognition. Generally, some embodiments of the present inventioncompute a set of SAR features from raw SAR data, and encode the SARfeatures into spiking sequences utilizing a spiking, multi-layerrecurrent neural network. The neural network converts spatialinformation of the SAR data into temporal sequences and enablesprocessing SAR data in the spiking domain, which is an energy efficienttechnique for data processing. The recurrent characteristic of theneural network further allows correlation of features over time.Averaged spiking rates of SAR data are used for target recognition by areadout neural layer trained through a supervised learning process.Simulated SAR data of civilian vehicles was used to evaluate the system,and experimental tests show that the proposed system is effective torecognize different SAR targets without forming SAR images. Someembodiments of the present invention can be implemented by aneuromorphic chip, thereby producing a SAR target recognition systemwith real-time computing capabilities and very low-power consumption.

Various portions of the target classification system that refer to theuse of a “processor” may be implemented with logic gates, or with anyother embodiment of a processing unit or processor. The term “processingunit” or “processor” is used herein to include any combination ofhardware, firmware, and software, employed to process data or digitalsignals. Processing unit hardware may include, for example, applicationspecific integrated circuits (ASICs), general purpose or special purposecentral processing units (CPUs), digital signal processors (DSPs),graphics processing units (GPUs), and programmable logic devices such asfield programmable gate arrays (FPGAs). While, in some embodiments, theinput spiking network 130, the multi-layer recurrent neural network 150,and the readout neural layer 170 of the target classification system areimplemented utilizing neuromorphic hardware, in some embodiments of thepresent invention, including some embodiments during a training processin which the parameters of the spiking input neural network 130 and themulti-layer recurrent neural network 150 are computed and the classifierweights of the readout neural layer 170 are computed, the spiking inputneural network 130, the multi-layer recurrent neural network 150, andthe readout neural layer 170 may be simulated by a processor.

An exemplary computer system 1200 in accordance with an embodiment isshown in FIG. 11. Exemplary computer system 1200 is configured toperform calculations, processes, operations, and/or functions associatedwith a program or algorithm. In one embodiment, certain processes andsteps discussed herein are realized as a series of instructions (e.g.,software program) that reside within computer readable memory units andare executed by one or more processors of exemplary computer system1200. When executed, the instructions cause exemplary computer system1200 to perform specific actions and exhibit specific behavior, such asdescribed herein.

Exemplary computer system 1200 may include an address/data bus 1210 thatis configured to communicate information. Additionally, one or more dataprocessing unit, such as processor 1220, are coupled with address/databus 1210. Processor 1220 is configured to process information andinstructions. In an embodiment, processor 1220 is a microprocessor.Alternatively, processor 1220 may be a different type of processor suchas a parallel processor, or a field programmable gate array.

Exemplary computer system 1200 is configured to utilize one or more datastorage units. Exemplary computer system 1200 may include a volatilememory unit 1230 (e.g., random access memory (“RAM”), static RAM,dynamic RAM, etc.) coupled with address/data bus 1210, wherein volatilememory unit 1230 is configured to store information and instructions forprocessor 1220. Exemplary computer system 1200 further may include anon-volatile memory unit 1240 (e.g., read-only memory (“ROM”),programmable ROM (“PROM”), erasable programmable ROM (“EPROM”),electrically erasable programmable ROM “EEPROM”), flash memory, etc.)coupled with address/data bus 1210, wherein non-volatile memory unit1240 is configured to store static information and instructions forprocessor 1220. Alternatively exemplary computer system 1200 may executeinstructions retrieved from an online data storage unit such as in“Cloud” computing. In an embodiment, exemplary computer system 1200 alsomay include one or more interfaces, such as interface 1250, coupled withaddress/data bus 1210. The one or more interfaces are configured toenable exemplary computer system 1200 to interface with other electronicdevices and computer systems. The communication interfaces implementedby the one or more interfaces may include wireline (e.g., serial cables,modems, network adaptors, etc.) and/or wireless (e.g., wireless modems,wireless network adaptors, etc.) communication technology.

In one embodiment, exemplar computer system 1200 may include an inputdevice 1260 coupled with address/data bus 1210, wherein input device1260 is configured to communicate information and command selections toprocessor 1220. In accordance with one embodiment, input device 1260 isan alphanumeric input device, such as a keyboard, that may includealphanumeric and/or function keys. Alternatively, input device 1260 maybe an input device other than an alphanumeric input device. In anembodiment, exemplar computer system 1200 may include a cursor controldevice 1270 coupled with address/data bus 1210, wherein cursor controldevice 1270 is configured to communicate user input information and/orcommand selections to processor 1220. In an embodiment, cursor controldevice 1270 is implemented utilizing a device such as a mouse, atrack-ball, a track-pad, an optical tracking device, or a touch screen.The foregoing notwithstanding, in an embodiment, cursor control device1270 is directed and/or activated via input from input device 1260, suchas in response to the use of special keys and key sequence commandsassociated with input device 1260. In an alternative embodiment, cursorcontrol device 1270 is configured to be directed or guided by voicecommands.

In an embodiment, exemplary computer system 1200 further may include oneor more optional computer usable data storage devices, such as storagedevice 1280, coupled with address/data bus 1210. Storage device 1280 isconfigured to store information and/or computer executable instructions.In one embodiment, storage device 1280 is a storage device such as amagnetic or optical disk drive (e.g., hard disk drive (“HDD”), floppydiskette, compact disk read only memory (“CD-ROM”), digital versatiledisk (“DVD”)). Pursuant to one embodiment, a display device 1290 iscoupled with address/data bus 1210, wherein display device 1290 isconfigured to display video and/or graphics. In an embodiment, displaydevice 1290 may include a cathode ray tube (“CRT”), liquid crystaldisplay (“LCD”), field emission display (“FED”), plasma display or anyother display device suitable for displaying video and/or graphic imagesand alphanumeric characters recognizable to a user.

Exemplary computer system 1200 is presented herein as an exemplarycomputing environment in accordance with an embodiment. However,exemplary computer system 1200 is not strictly limited to being acomputer system. For example, an embodiment provides that exemplarycomputer system 1200 represents a type of data processing analysis thatmay be used in accordance with various embodiments described herein.Moreover, other computing systems may also be implemented. Indeed, thespirit and scope of the present technology is not limited to any singledata processing environment. Thus, in an embodiment, one or moreoperations of various embodiments of the present technology arecontrolled or implemented utilizing computer-executable instructions,such as program modules, being executed by a computer. In one exemplaryimplementation, such program modules include routines, programs,objects, components and/or data structures that are configured toperform particular tasks or implement particular abstract data types. Inaddition, an embodiment provides that one or more aspects of the presenttechnology are implemented by utilizing one or more distributedcomputing environments, such as where tasks are performed by remoteprocessing devices that are linked through a communications network, orsuch as where various program modules are located in both local andremote computer-storage media including memory-storage devices.

While the present invention has been described in connection withcertain exemplary embodiments, it is to be understood that the inventionis not limited to the disclosed embodiments, but, on the contrary, isintended to cover various modifications and equivalent arrangementsincluded within the spirit and scope of the appended claims, andequivalents thereof.

What is claimed is:
 1. A method for identifying a target in a syntheticaperture radar signal, the method comprising: extracting, by a featureextractor, a plurality of features directly from phase history data ofthe synthetic aperture radar signal; encoding, by an input spikingneural network, the features as a first plurality of spiking signals;supplying the spiking signals to a multi-layer recurrent neural networkto compute a second plurality of spiking signals; computing, by areadout neural layer, a signal identifier based on the second pluralityof spiking signals; and outputting the signal identifier from thereadout neural layer, the signal identifier identifying the target. 2.The method of claim 1, wherein the plurality of features comprise anamplitude of the synthetic aperture radar signal.
 3. The method of claim1, wherein the plurality of features comprise a phase of the syntheticaperture radar signal.
 4. The method of claim 1, wherein the readoutneural layer comprises a linear classifier.
 5. The method of claim 1,wherein the input spiking neural network, the multi-layer recurrentneural network, and the readout neural layer are implemented by aneuromorphic chip.
 6. The method of claim 1, further comprisingcomputing average spiking rates from the second plurality of spikingsignals, wherein the signal identifier is computed based on the averagespiking rates.
 7. The method of claim 1, wherein the multi-layerrecurrent neural network comprises: a first excitatory neuron layerconfigured to receive the first plurality of spiking signals from theinput spiking neural network; a first inhibitory neuron layer; a secondexcitatory neuron layer; and a second inhibitory neuron layer connectedto the second excitatory neuron layer, the first excitatory neuron layerbeing configured to supply spiking signals to: the first excitatoryneuron layer; the first inhibitory neuron layer; and the secondexcitatory neuron layer, the first inhibitory neuron layer beingconfigured to supply spiking signals to the first excitatory neuronlayer, the second excitatory neuron layer being configured to supplyspiking signals to: the second excitatory neuron layer; the secondinhibitory neuron layer; and the readout neural layer, and the secondinhibitory neuron layer being configured to supply spiking signals tothe second excitatory neuron layer.
 8. The method of claim 7, whereinthe input spiking neural network comprises a plurality of input neuronsarranged in a grid, wherein the first excitatory neuron layer comprisesa plurality of first excitatory neurons arranged in a grid, theplurality of first excitatory neurons comprising a plurality of criticalneurons uniformly distributed in the grid, and wherein the plurality ofinput neurons are connected to the critical neurons to maintain spatialrelationships between the input neurons in corresponding ones of thecritical neurons.
 9. The method of claim 7, wherein the first excitatoryneuron layer comprises a plurality of excitatory neurons arranged in agrid, and wherein a neuron of the first excitatory neuron layer isconfigured to supply spiking signals to neurons in a local neighborhoodof the grid around the neuron.
 10. The method of claim 7, wherein thefirst excitatory neuron layer comprises a plurality of first excitatoryneurons arranged in a grid, wherein the second excitatory neuron layercomprises a plurality of second excitatory neurons arranged in a grid,and wherein the plurality of first excitatory neurons are connected tothe second excitatory neurons to maintain spatial relationships betweenthe first excitatory neurons in corresponding ones of the secondexcitatory neurons.
 11. A system configured to identify a target in asynthetic aperture radar signal, the system comprising: a featureextractor configured to extract a plurality of features directly fromphase history data of the synthetic aperture radar signal; an inputspiking neural network configured to encode the features as a firstplurality of spiking signals; a multi-layer recurrent neural networkconfigured to compute a second plurality of spiking signals based on thefirst plurality of spiking signals; a readout neural layer configured tocompute a signal identifier based on the second plurality of spikingsignals; and an output configured to output the signal identifier, thesignal identifier identifying the target.
 12. The system of claim 11,wherein the plurality of features comprise an amplitude of the syntheticaperture radar signal.
 13. The system of claim 11, wherein the pluralityof features comprise a phase of the synthetic aperture radar signal. 14.The system of claim 11, wherein the readout neural layer comprises alinear classifier.
 15. The system of claim 11, wherein the input spikingneural network, the multi-layer recurrent neural network, and thereadout neural layer are implemented by a neuromorphic chip.
 16. Thesystem of claim 11, wherein the readout neural layer is configured tocompute average spiking rates from the second plurality of spikingsignals, wherein the signal identifier is computed based on the averagespiking rates.
 17. The system of claim 11, wherein the multi-layerrecurrent neural network comprises: a first excitatory neuron layerconfigured to receive the first plurality of spiking signals from theinput spiking neural network; a first inhibitory neuron layer; a secondexcitatory neuron layer; and a second inhibitory neuron layer connectedto the second excitatory neuron layer, the first excitatory neuron layerbeing configured to supply spiking signals to: the first excitatoryneuron layer; the first inhibitory neuron layer; and the secondexcitatory neuron layer, the first inhibitory neuron layer beingconfigured to supply spiking signals to the first excitatory neuronlayer, the second excitatory neuron layer being configured to supplyspiking signals to: the second excitatory neuron layer; the secondinhibitory neuron layer; and the readout neural layer, and the secondinhibitory neuron layer being configured to supply spiking signals tothe second excitatory neuron layer.
 18. The system of claim 17, whereinthe input spiking neural network comprises a plurality of input neuronsarranged in a grid, wherein the first excitatory neuron layer comprisesa plurality of first excitatory neurons arranged in a grid, theplurality of first excitatory neurons comprising a plurality of criticalneurons uniformly distributed in the grid, and wherein the plurality ofinput neurons are connected to the critical neurons to maintain spatialrelationships between the input neurons in corresponding ones of thecritical neurons.
 19. The system of claim 17, wherein the firstexcitatory neuron layer comprises a plurality of excitatory neuronsarranged in a grid, and wherein a neuron of the first excitatory neuronlayer is configured to supply spiking signals to neurons in a localneighborhood of the grid around the neuron.
 20. The system of claim 17,wherein the first excitatory neuron layer comprises a plurality of firstexcitatory neurons arranged in a grid, wherein the second excitatoryneuron layer comprises a plurality of second excitatory neurons arrangedin a grid, and wherein the plurality of first excitatory neurons areconnected to the second excitatory neurons to maintain the spatialrelationships between the first excitatory neurons in corresponding onesof the second excitatory neurons.
 21. A system for identifying a targetin a synthetic aperture radar signal, the system comprising: means forextracting a plurality of features directly from phase history data ofthe synthetic aperture radar signal; means for encoding the features asa first plurality of spiking signals; means for supplying the spikingsignals to a multi-layer recurrent neural network to compute a secondplurality of spiking signals; means for computing a signal identifierbased on the second plurality of spiking signals; and means foroutputting the signal identifier from the means for computing the signalidentifier, the signal identifier identifying the target.